An image can be represented by a two-dimensional array of pixels. Each row of pixels is referred to as a scan line. The value assigned to each pixel determines its intensity and/or color when the image is recreated. The resolution of the image is represented by the size of the two-dimensional array, i.e., the number of pixels. The greater the number of pixels, the higher the resolution and vice versa. A low-resolution, i.e., standard definition (SD), image can be enhanced to become a high-resolution, i.e., high definition (HD), image, by increasing the number of scan lines, which in turn increases the number of pixels. The pixels in the additional scan lines are referred to as “missing” pixels because their values must be determined from something other than the live source that was captured in the original image.
The simplest ways to generate values for missing pixels is by line repetition, i.e., repeating a scan line, or line averaging, i.e., averaging the two vertically adjacent scan lines. These techniques increase the number of scan lines and is adequate if the image is relatively smooth with no high spatial frequency components in the vertical direction. Nevertheless, if the image has fine details in the vertical direction and/or the image has sharp diagonal edges, the enhanced image will exhibit annoying and undesirable visual artifacts such as, for example, blur in the fine details and aliasing along the diagonal edges. The aliasing causes the diagonal edge to appear “jaggy,” hence the reference to “jaggy edges.”
Alternatively, a missing pixel value can be generated by analyzing available pixel values of pixels surrounding the missing pixel and performing complex calculations to generate an interpolated value for the missing pixel. This process, while effective in generating an enhanced image with significantly no visual artifacts, typically takes a long time and requires a high level of processing power. For instance, the analysis and computation usually takes several seconds and is performed by a central processing unit (CPU). Generally, this process can be feasible for printers or scanners. Nevertheless, for a system that requires a shorter conversion time, i.e., fractions of a second, and that does not typically use a CPU, this technique is not feasible. For example, this technique would not be feasible for a television display system that has a display frame rate typically in the range of about 60 (or more) frames per second and typically uses an application-specific integrated circuit (ASIC).
A less calculation intensive and faster technique for generating a pixel value for a missing pixel includes calculating a weighted sum of available pixel values surrounding the missing pixel and using a set of interpolation filter coefficients. Nevertheless, if the missing pixel value is interpolated using a non-adaptive interpolation filter, such as a bilinear filter, a quadratic filter, a cubic filter, a sinc filter, or other similar filters, it can be difficult to obtain a HD output image because the input SD image does not have high-frequency components that an output HD image should have. Accordingly, for these non-adaptive filtering methods, the annoying visual artifacts mentioned above can be present in the enhanced image.
The missing high-frequency components can be restored by using a linear combination of the SD image and one of a plurality of adaptive interpolation filter coefficient sets. Each set of adaptive interpolation filter coefficients can be pre-determined and associated with a particular interpolation direction. The value for the missing pixel can be generated by selecting one of a plurality of pre-determined interpolation directions and then using the associated set of interpolation filter coefficients to execute the interpolation. This approach, however, may fail to reduce aliasing along a diagonal edge when the diagonal edge is along a direction that is not close to any one of the pre-determined interpolation directions or when multiple dominant edge directions are present close to the missing pixel. In these instances, the interpolated value for each missing pixel along the diagonal edge can be wrong, and the resulting enhanced image can exhibit unusual and annoying visual artifacts such as those mentioned above.
In another approach, the missing high-frequency components can be restored by classifying the pixels surrounding the missing pixel and using the classification to select one or more sets of interpolation filter coefficients from a look-up table to execute the interpolation. While effective in producing an enhanced image with little or no visual artifacts, this approach requires high processing power to perform the complex computations to determine the classification and to perform the interpolation process, and also requires large amounts of memory for storing the large look-up tables corresponding to the pre-determined sets of interpolation filter coefficients. Such processing power and memory requirements are expensive. In addition, the time required to perform the interpolation for each missing pixel in a frame is generally too long for certain display systems, such as televisions.
Accordingly there exists a need for an improved adaptive image data interpolation process that generates an enhanced, i.e., HD, image from an SD image. The enhanced image should be substantially free of annoying visual artifacts, such as jaggy edges, and the process should not require excessive memory or computational resources.